Iteratively improving an advertisement response model

ABSTRACT

There are provided systems and methods for iteratively improving an advertisement response model. A payment provider may perform operations that include training an advertisement response model using a training data set. The operations include determining that a first accuracy value corresponding to the advertisement response model is less than a accuracy value threshold. The operations further include identifying, based on executing the advertisement response model using a target data set that is different from the training data set, one or more units from the target data set for which to run the advertising campaign. The operations also include receiving one or more responses corresponding to a run of the advertising campaign with respect to the identified one or more units from the target data set and updating the training data set based on the one or more responses. The operations further include training an advertisement response model using resulting training data and repeating the operations as long as the accuracy value of the resulting model stays below the threshold or until the increase in the accuracy value with each iteration becomes unprofitable with respect to the costs of acquiring responses from further units from the target dataset.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a continuation-in-part of U.S. patentapplication Ser. No. 14/961,727, filed Dec. 7, 2015.

TECHNICAL FIELD

The present application generally relates to predictive analysis usingwebsite data and more specifically to predicting merchant behavior usingmerchant website terms.

BACKGROUND

An online service provider may offer various services to one or moremerchants in connection with operation of online websites or mobileapplications, such as processing payments to the merchants by consumers.For example, the payment services may be used to provide a payment tothe merchant for a transaction using a consumer's payment account withthe payment provider or other payment instrument (e.g., a credit/debitcard, a banking account, a gift card, etc.). The payment provider mayalso assist the merchant with other services, such as checkout andshopping services. Such features may be implemented in a website and/ordedicated application of the merchant through software development kitsand/or other processes offered by the payment provider. However, inorder to implement these systems into a merchant's website orapplication, the payment provider and merchant must be aware of eachother and their corresponding systems. For example, the merchant must beaware that the payment provider offers certain payment processes.Moreover, in order to provide targeted advertising to the merchant ofthe payment provider's services that would be ideal for use in themerchant's shopping experience, the payment provider must have knowledgeof the merchant's current systems and offerings. While generaladvertising by the payment provider may assist the payment provider isreceiving more brand recognition, merchants may still be unaware of allof the features of the payment provider that may optimize the merchant'sshopping experience to consumers. Furthermore, merchants may not beaware of how to change their website or consumer experience to increaseeffectiveness of the website.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a networked system suitable forimplementing the processes described herein, according to an embodiment;

FIG. 2 is a representation of two example merchant website interfacesfor use in determining a dictionary of website terms and providingtargeted advertisement of payment services, according to an embodiment;

FIG. 3 is an exemplary system environment having merchant websiteservers with data necessary for determining a dictionary of websiteterms and providing targeted advertisement of payment services,according to an embodiment;

FIG. 4 is a flowchart of an example process for predicting merchantbehavior using merchant website terms, according to an embodiment;

FIG. 5 is a block diagram of a networked system suitable for improvingan advertisement response model, according to a particular embodiment;

FIG. 6 is a flow chart that illustrates a method for improving anadvertisement response model, according to a particular embodiment; and

FIG. 7 is a block diagram of a computer system suitable for implementingone or more components in FIG. 1, according to an embodiment.

Embodiments of the present disclosure and their advantages are bestunderstood by referring to the detailed description that follows. Itshould be appreciated that like reference numerals are used to identifylike elements illustrated in one or more of the figures, whereinshowings therein are for purposes of illustrating embodiments of thepresent disclosure and not for purposes of limiting the same.

DETAILED DESCRIPTION

Provided are methods utilized for predicting merchant results usingmerchant website terms. Systems suitable for practicing methods of thepresent disclosure are also provided.

A service provider, such as a payment provider, may offer services tovarious merchants for use by the merchants to provide a shoppingexperience to consumers. Various services offered to merchants mayinclude shopping interfaces and/or marketplaces that may assist the userin finding one or more products, goods, and/or services (referred toherein as an “item” or “items”) for sale by the merchant. The processesmay further include checkout and payment processes that may be used toselect items for purchase, generate a transaction for the items, providepayment for the items, and arrange delivery, pick-up, or other retrievalof the items by the user. Such processes may be offered by back-endprocessing systems of the payment provider (e.g., server and/or databasecomponents and processing features), as well as features that may beimplemented into a website or dedicated application of the merchantthrough software and other processes or information provided by thepayment provider. In other embodiments, different service providers mayalso use similar processes as those described herein to provide variousservices to merchants or other entities.

The payment provider or other service provider may wish to advertise orotherwise market their services to existing and/or prospective customersof the payment/service provider. For example, customers of a paymentprovider may correspond to merchants utilizing the payment provider'sservices. In order to determine which merchants to advertise the paymentprovider's services and systems to, the payment provider may processwebsite data for the payment provider's current merchant customers. Thewebsite data may correspond to a website of the merchant having text,images, and other data represented on the web site of the merchant. Invarious embodiments, the payment provider may also further use datataken from a dedicated application (e.g., mobile device application) ofthe merchant. The payment provider may thus scan the websites (and/ordedicated applications) of the current merchant customers for thepayment provider to pull the website data for processing. Once pulled,the payment provider may parse the data to determine text and otherterms included on the merchant websites (and/or dedicated applications).The terms may include terms used by the merchant on their website, suchas sales information for one or more items, payment and checkoutinformation and process descriptions, and/or other merchant informationappearing on the websites. The terms may also include text andinformation entered by users (e.g., the merchant's customers) to thewebsite, including product reviews, forum postings, messages, servicerequests, and other types of information submitted by users to themerchant's website. The terms may correspond to a single text element,such as a character or word, as well as multiple text elements (e.g., asentence). In various embodiments, the terms may further include visualimages appearing on the merchant website, as well as logos and othergraphics. Such images may be processed using image processing techniquesto determine the content of the image, or may be matched to pre-existingimages, such as similar logos, banners, and graphics used by merchants,payment providers, customer users, and/or service providers.

Once processed, the payment provider may determine a dictionary of termsoccurring on the merchants' websites. The dictionary may includerankings of how often terms appear on the websites, on a percentage ofthe websites that the terms appear, and/or other analytic associatedwith the appearance of the terms on the merchant websites. The terms mayprovide an indication of the term's importance or prevalence ofappearance on the merchant websites. For example, a term A may occur onevery merchant website so that the term A provides a good indicationthat a website having the term would also be a merchant and potentialcustomer of the payment provider. Similarly, a term B may occur multipletimes on a medium to high percentage of the merchant websites (e.g.,75%) so that the term B also indicates that other website may be apotential merchant customer of the payment provider. In contrast, a termappearing only a few times and/or on a low percentage of the websitesmay not be indicative of a potential merchant customer of the paymentprovider. Thus, the number of times the term appears on a single websiteand across a plurality of merchant's websites may be important. In otherembodiments, other data for the term may also be used to score or rankthe term, or may provide other indications of how well the term performsin predicting behavior of other prospective merchant clients based ontheir website data. For example, the placement of a term on the websitemay be important (e.g., on the front webpage of a website or a sub-pageof the website), a size or font of the term (e.g., terms with large sizeor prominent fonts so as to be visible), a color of the term, and/or aratio of the term to other terms and elements on the website (e.g., ifthe term appears in a high ratio or where the term may appear in a lowratio due to a high volume of text and other terms presented on thewebsite). The website data may also be pulled from previous merchantcustomers of the payment provider.

The payment provider may also determine a total payment volume by themerchant, which may correspond to a number, amount, and/or frequency ofpayments to the merchant by users (e.g., customers of the merchant). Thetotal payment volume may be used to determine rankings for the merchantsand how often the merchants use the payment provider or how much of thepayment provider's systems the merchants use. The payment provider mayalso review each individual merchant to determine the payment processingservices and other services/features of the payment provider that themerchant uses. Using this data, the payment provider may also correlatethe terms with types of merchants and/or payment services used by themerchants. For example, a term C may be used on most or all of themerchant websites corresponding to merchants having a high total paymentvolume or a payment volume over a certain set threshold. Thus, theoccurrence of term C on another new website may indicate that the newwebsite may behavior similarly to the websites of current merchantcustomers of the payment provider and therefore have a similar totalpayment volume. In further embodiments, a term D may appear on most orall of the websites of current merchant customers of the paymentprovider that utilize a payment process A for transactions and paymentprocessing. Thus, another new website having term D may indicate thatthe other new website may similarly wish to use the payment process A asthe current merchant customers of the payment provider.

In various embodiments, the payment provider may update the dictionaryof terms and their corresponding information (e.g., occurrence,importance, score, associated total payment volume, and/or associatedpayment processes) based on new information retrievable by the paymentprovider. For example, the payment provider may scan the current and/orprevious merchant customers' websites to determine new terms and/orupdate terms already in the dictionary. The payment provider may alsoadd new and/or prospective merchant websites into the dictionary,particularly if the prospective merchant becomes a customer of thepayment provider (e.g., purchases and/or implements the paymentprovider's processes). The dictionary may also use historicalinformation for the websites and/or with the payment provider, which mayinclude when merchants became a customer of the payment provider,updates to websites and added/removed content, and other data. The termusage may be country and/or language specific, and may be weigheddifferently based on the country/language. For example, a term D may bemore important or prevalent to English websites in one country overanother. However, as processing of the terms depends only on theiroccurrence, importance, and/or associated total payment volume/paymentprocesses, the dictionary generated using the website data may beindependent of word definitions and language semantics, and may alsofunction in any language through processing of term appearance,importance, and/or merchant analytics.

Once the dictionary of terms is determined using the websites of currentmerchant customers for the payment provider, the payment provider mayretrieve website data from prospective merchants. In variousembodiments, a merchant may wish to know which payment processes areideal for the merchant based on the website data for past and currentmerchants using the payment provider. Thus, the prospective merchant mayrequest that the payment provider scan the website of the prospectivemerchant. However, in other embodiments, the payment provider mayretrieve the information by scanning the Internet for prospectivemerchant websites based on the presence of words in the dictionary.

Once merchant website data for a prospective merchant is retrieved, themerchant website data may be processed using the dictionary of terms todetermine whether any terms appearing on the prospective merchant'swebsite match the terms in the dictionary. For example, the paymentprovider may perform matching on the terms on the prospective merchant'swebsite (e.g., front webpage and/or sub-webpages constituting themerchant website) to the terms in the dictionary of terms. Thus, thepayment provider may perform analytics on the prospective merchant andpredict behavior of the merchant based on the appearance of terms on theprospective merchant's website that match the terms in the dictionary.The payment provider may determine whether the merchant may have a hightotal payment volume based on the appearance of one or more termsassociated with merchants having a high total payment volume on thewebsite of the merchant. In other embodiments, the payment provider maydetermine payment services of the payment provider or other servicesoffered by the payment provider that the prospective merchant may wishto implement in their website based on services used by merchants havingmatching website terms to the terms on the website of the prospectivemerchant. The occurrence of terms on the prospective merchant's websitemay also be scored against the dictionary of terms. For example, thenumber of appearances, the ratio appearances to other text and terms,the size, the color, the placement, and/or other information for theterm may all be used to score how important the term is and howprevalent the term appears on the prospective merchant's website.

In various embodiments, a dedicated application of the prospectivemerchant may instead be processed against a dictionary of terms fordedicated applications of past and current merchant customers of thepayment provider. Thus, the algorithm to determine the dictionary ofterms may be different depending on whether the payment provider wishesto create a dictionary for mobile applications, websites, or otherelectronic merchant portals. In this regard, the algorithm may judge thetechnology behind the interfaces presented (e.g., the websites,applications, and/or other interfaces), for example, by weighingkeywords and other scraped information depending on the type ofinformation. The analytics and weighing for a word or other term may bedifferent for the same term based on whether the term appears on anapplication interface or a website interface. Thus, the dictionary maybe platform specific and rely on terms used within the application orwebsite, or may be platform agnostic and rely instead on term usage. Theprospective merchant may be determined based on the dictionary of termsspecific to the type of interfaces processed for the prospectivemerchant.

In various embodiments, the dictionary of terms may also be subdividedto a type of merchant, items sold, size of merchant, or other relevantmerchant metric. For example, the dictionary of terms may includegroupings of large merchants, merchants who sell a certain type of good(e.g., consumer electronics), local merchants, national/worldwidemerchants, service providers, digital goods providers, physical itemsproviders, etc. Using the grouping of the merchant, the dictionary ofterms may be specific to sub-types of merchants, and may be used todetermine the prospective merchant matching characteristics of thesub-type of merchant. For example, the payment provider may search forprospective merchants matching large merchants, electronic devicemerchants, etc. The payment provider may then determine the prospectivemerchant is that specific type of merchant, and may tailor analytics andadvertisements to that type of merchant.

Once the prospective merchant is determined, the prospective merchant'stotal payment volume, and/or services for use by the prospectivemerchant, the payment provider may determine one or more advertisementsor other messages to communicate to the prospective merchant. Themessage(s) may generally advertise the payment provider's services tothe merchant. In other embodiments, the message(s) may include targetedadvertisements, which may include information on predictive analysis ofthe merchant as compared to other merchants having similar websiteterms. For example, the message(s) may include information on the typeof payments the merchant may be expected to receive through the web siteand/or the type of payment processing required by the merchant based onthe similar merchants from the dictionary of terms. Thus, the message(s)may advertise the payment provider's services and other products thatmay be ideal, best, or most preferable in handling these types ofpayments or other required services. In other embodiments, themessage(s) may include advertisements for payment processes used by thesimilar merchants determined from the prospective merchant's websiteterms and the dictionary of terms. In still further embodiments, themessage(s) may include behavioral and predictive analytics to themerchant based on the behavior and other analytics of the past andcurrent merchant customers of the payment provider having matchingwebsite terms to the website of the prospective merchant. In otherembodiments, the payment provider may instead provide a message to acurrent and/or prospective merchant, where the message alerts themerchant of analytics of terms and term information for similarmerchants. The payment provider may choose the information for thesimilar merchants based on a success rating of the merchant (e.g., totalpayment volume) or other metric in order to alert the prospective orcurrent merchant of better practices. In various embodiments, thepayment provider can suggest to a current customer/merchant that certainwords/terms should be changed, moved, etc.

The merchant may respond to the advertisement, such as by declining topurchase the service, enroll in the service, or use the paymentprovider. Further messages to the prospective merchant and otherprospective merchant's may be tailored to the response, for example, byremoving advertised material that the prospective merchant respondedpoorly to or declined. The further messages and advertisements mayprovide the next best services based on the merchant's total paymentvolume and/or similar merchants. Where the prospective merchant acceptsthe offer and utilizes a service of the payment provider, the paymentprovider may assist the merchant in implementing the service into themerchant's website and/or dedicated application. Moreover, the paymentprovider may update the dictionary with information for the prospectivemerchant and the prospective merchant's response, who has now become acurrent merchant customer to the payment provider.

According to other embodiments, the payment provider may also beconfigured to improve an advertisement response model. Given a targetdata set, the advertisement response model may be configured to predictthe responses of units included in the target data set to a particularadvertising campaign. As used herein, a “unit” of a data set maycorrespond to a user of the payment provider, a prospective user of thepayment provider, or any prospective customer, including but not limitedto, consumers and merchants and prospective consumers and merchants.Further, a “target data set” may include units whose responses to theadvertising campaign are unknown. A “training data set” may includeunits whose responses to the advertising campaign are known.

Based on predicted responses output by the advertisement response model,the payment provider may target particular units in the target data set(e.g., units with positive predicted responses) for the advertisingcampaign. Further, by applying a model enhancement process described inmore detail below, the payment provider may be able to improve theadvertisement response model. For example, the model enhancement processmay improve the advertisement response model's accuracy in predictingthe responses of the units included in the target data set. Varioustypes of advertisement response models may be improved by the modelenhancement process including, but not limited to classificationalgorithms such as AdaBoost classifier, logistic regression, supportvector machines, gradient boosting classifier, Isolation Forest, lineardiscriminant analysis, neural networks (e.g., multi-layer perceptron),decision trees classifier, Random Forest classifier, naïve Bayesianclassifier, and Bagging classifier.

In some examples, the model enhancement process begins with the paymentprovider executing the advertisement response model using a trainingdata set. In other words, the advertisement response model may betrained based on the training data set. The training data set mayinclude one or more units, and the training data set may also indicateknown responses to the advertising campaign for each unit of the one ormore units (e.g., how each unit responds to the particular advertisingcampaign). In some cases, responses may be categorized by responsetypes. For instance, the response types may include a positive response(e.g., a user or potential user buys or would buy the product or serviceoffered by the advertising campaign) or a negative response (e.g., auser or potential user does not or would not buy the product or serviceoffered by the advertising campaign). However, it will be appreciatedthat in other embodiments, any number of response types (including moreor fewer response types) may be indicated by the training data set.

In certain embodiments, the training data set may be a previouslyexisting training data set that is provided to the advertisementresponse model. In other embodiments, the training data set may begenerated as part of the model enhancement process. To this end, varioustechniques and methods may be used in order to develop the training dataset. For example, the training data set may be generated based on a runof the advertising campaign with respect to a random sampling of aparticular population. As such, the responses of each unit of the randomsampling of the population to the advertising campaign may be stored inthe training data set. It will be appreciated that other methods ofgenerating the training data set are also contemplated.

Subsequent to training the advertisement response model using thetraining data set, the payment provider may determine an accuracy valuecorresponding to the advertisement response model. In certainembodiments, the accuracy value may be calculated as part of executingthe advertisement response model. The accuracy value may indicate anaccuracy of the advertisement response model in predicting the responseof units in a particular data set to the advertising campaign. Accordingto a particular embodiment, the accuracy value may include numbers orratios. In some embodiments, the accuracy value may correspond to aratio. For example, the advertisement response model may predict aparticular data set to have a positive outcome with respect to theadvertising campaign. The accuracy value may be a ratio of the number ofunits in the particular data set that actually have a positive outcomeas a result of running the advertising campaign versus the total numberof units in the subset of the particular data set that the model haspredicted to have the positive outcome.

In certain implementations the payment provider may determine that theaccuracy value is less than a accuracy value threshold. In response todetermining that the accuracy value is less than the accuracy valuethreshold, the payment provider may execute the advertisement responsemodel using the target data set. The target data set may be differentthan the training data set, and in some cases, may be significantlylarger than the training data set. Based on the execution of theadvertisement response model using the target data set, the paymentprovider may calculate a certainty score corresponding to each unit ofthe target data set. For example, in certain embodiments, the certaintyscores may be an output of executing of the advertisement response modelusing the target data set.

Based on the certainty scores, the payment provider may select a subsetof the target data set. For example, the payment provider may select thesubset based on the subset having relatively low certainty scores. Thelow certainty scores may indicate that the advertisement response modelis relatively uncertain with respect to predicting how the units in thesubset would respond to the advertising campaign. According to aparticular embodiment, the payment provider may rank the units of thetarget data set according to their respective certainty scores. As such,the selected subset may include a predetermined number of lowest rankedunits in the target data set (e.g., the units having the lowestcertainty scores). For example, the payment provider may select 200units with the lowest certainty scores to be included in the subset.According to another particular embodiment, the payment provider mayselect the subset to include any unit having a corresponding certaintyscore that is below a certainty score threshold.

Further, a run of the advertising campaign may be performed using theselected subset of the target data set. Based on the run, one or moreresponses to the advertising campaign may be associated with the unitsin the subset and provided to the payment provider. Since the units inthe subset are now associated with known responses, the payment providermay update the training data set to include the subset the associatedresponses. The updated training data set is therefore augmented bygathering actual responses to advertising campaign from units in thesubset—units whose predicted responses the advertisement response modelwas previously uncertain about. Additionally, subsequent to receivingthe responses from the subset of the target data set, the paymentprovider may remove the units of the subset from the target data set.

Using the updated training data set, the model enhancement process maythen be repeated beginning with the payment provider executing theadvertisement response model using the updated target training data set.Moreover, the model enhancement process may be repeated or iterateduntil a calculated accuracy value for the advertisement response modelreaches or is above a accuracy value threshold. Thus, the training dataset is continually improved, thereby also improving the advertisementresponse model until the accuracy value threshold is reached. Subsequentto determining that the calculated accuracy value reaches or exceeds theaccuracy value threshold, the payment provider may execute theadvertisement response model using the remaining units in the targetdata set (e.g., the target data set minus any subsets that have beenremoved from the target data set during iterations(s) of the modelenhancement process). Based on execution of the advertisement responsemodel using the remaining units in the target data set, the paymentprovider may determine which of the remaining units in the target dataset have a positive predicted response to the advertising campaign. Theadvertising campaign may then be run to target the determined units thathave a positive predicted response.

As shown above, using the model enhancement process, the advertisementresponse model may be iteratively improved. Subsets of the target dataset, whose predicted responses (to the advertising campaign) theadvertisement response model is relatively uncertain about, may beidentified. Uncertainty about the responses of the units in the subsetis mitigated by running the advertising campaign with respect to theseunits. From running the advertising campaign, the units' actualresponses to the advertising campaign may be obtained. The units andtheir corresponding responses are iteratively added to an initialtraining data set, which is used to iteratively train the advertisementresponse model. In this manner, the accuracy of the advertisementresponse model is improved until an accuracy threshold is reached.

As a result, the model enhancement process describe above may reduce atotal cost of managing the advertising campaign with respect to thetarget data set. For example, transmitting advertisements to differentunits (e.g., user devices and potential user devices) and obtainingresponses to the advertisements may be relatively expensive to perform.Further, a particular unit that does not respond positively to atransmitted advertisement (e.g., does not buy the product or serviceoffered by the advertisement), drives up the cost of transmitting the adeven further. By improving the accuracy value of the advertisementresponse model, the conversion rate of running the advertising campaignwith respect to the units selected by the advertisement response modelmay also improve. Such improvement in the conversion rate may more thanoffset any costs associated with improving the training data set duringeach iteration of the model enhancement process, thereby reducing thetotal cost of managing the advertising campaign. The reduced costs canlead to a new breakeven point for the profitability of certainadvertising campaigns. Thus, model enhancement process may make certaincampaigns profitable that would otherwise have been unprofitableotherwise. Additionally, by enabling the advertisement response model toreach more customers that are interested in the products or servicesbeing advertised, customer satisfaction for the merchant and/or entityrunning the advertising campaign may be improved.

According to certain embodiments, the payment provider may select aparticular advertisement response model for improvement based ontheoretical considerations (e.g., known performance of the particularadvertisement response model in similar situations, robustness),practical considerations (e.g., data availability, computationalcomplexity of the particular advertisement response model, and/orinterpretability of the particular advertisement response model), and/ordata driven considerations. For example, the accuracy value of theadvertisement response model and the number of advertised units (e.g.,the number of units from the target data set for which the advertisingcampaign has been run) to achieve the accuracy value may be the primaryfactors for determining the profitability of the advertising campaign.These parameters (e.g., the accuracy value and the number of advertisedunits) can be estimated on an existing dataset (e.g., from pastadvertising campaigns) where the outcomes of the past advertisingcampaigns are already known. The payment provider can also measure theparameters dynamically during an advertising. For example, multipleadvertisement response models can be used to drive a single advertisingcampaign in parallel and as a particular model or a group of the modelsis showing better results relative to the other, the particular model orgroup of models may be provided a larger share of data from both thetraining data set and the target data set.

FIG. 1 is a block diagram of a networked system 100 suitable forimplementing the processes described herein, according to an embodiment.As shown, system 100 may comprise or implement a plurality of devices,servers, and/or software components that operate to perform variousmethodologies in accordance with the described embodiments. Exemplarydevices and servers may include device, stand-alone, andenterprise-class servers, operating an OS such as a MICROSOFT® OS, aUNIX® OS, a LINUX® OS, or other suitable device and/or server based OS.It can be appreciated that the devices and/or servers illustrated inFIG. 1 may be deployed in other ways and that the operations performedand/or the services provided by such devices and/or servers may becombined or separated for a given embodiment and may be performed by agreater number or fewer number of devices and/or servers. One or moredevices and/or servers may be operated and/or maintained by the same ordifferent entities.

System 100 includes a user, a current merchant server 110, a prospectivemerchant server 130, and a payment provider server 150 in communicationover a network 180. Current merchant server 110 may be established by amerchant customer (not shown) of payment provider server 150 that setsup current merchant server 110 to provide a website (and/or dedicatedmerchant application) to consumers that may purchase items with themerchant. For example, the website may function as a marketplace orother sales interface for consumers to view items and item information,select items for purchase, enter a checkout process, and pay themerchant. In order to provide payments, current merchant server 110 mayutilize one or more processes provided by payment provider server 150.Payment provider server 150 may determine a dictionary of termsappearing on the website of current merchant server 110 and/or othermerchant servers, and may associate the terms with other information forthe website and/or merchant for current merchant server 110. Using thedictionary, payment provider server 150 may identify prospectivemerchant server 130 having a website for a prospective merchant customer(not shown). Payment provider server 150 may utilize the dictionary toprovide various features and services to prospective merchant server130.

Current merchant server 110, prospective merchant server 130, andpayment provider server 150 may each include one or more processors,memories, and other appropriate components for executing instructionssuch as program code and/or data stored on one or more computer readablemediums to implement the various applications, data, and steps describedherein. For example, such instructions may be stored in one or morecomputer readable media such as memories or data storage devicesinternal and/or external to various components of system 100, and/oraccessible over network 180.

Current merchant server 110 may be implemented using any appropriatehardware and software configured for wired and/or wireless communicationwith communication device 130 and/or payment provider server 150.Current merchant server 110 may correspond to a device, server, or cloudcomputing architecture to provide sales of items, for example, through aphysical merchant location and/or an online marketplace accessible overa network connection that has a corresponding physical location forretrieval of one or more sold items. Current merchant server 110 mayfurther be used to process payments for items, provide incentives forpurchase of items, advertise items, and/or allow release of items tocurrent merchant server 110. Although a merchant device is shown, themerchant device may be managed or controlled by any suitable processingdevice. Although only one merchant device is shown, a plurality ofmerchant devices may function similarly.

Current merchant server 110 of FIG. 1 contains a website application120, a sales application 112, other applications 114, a database 116,and a communication module 118. Website application 120, salesapplication 112, and other applications 114 may correspond to processes,procedures, and/or applications executable by a hardware processor, forexample, a software program. In other embodiments, current merchantserver 110 may include additional or different modules havingspecialized hardware and/or software as required.

Website application 120 may correspond to one or more processes toexecute modules and associated specialized hardware of current merchantserver 110 to provide a website for a merchant, which may include one ormore webpages accessible over a network by a consumer in order to shopwith the merchant. In this regard, website application 120 maycorrespond to specialized hardware and/or software of current merchantserver 110 that provide a website and corresponding interfaces. Thewebsite may correspond to a merchant marketplace for the currentmerchant associated with current merchant server 110, or other salesinterface. Website application 120 may provide the website over anetwork, for example, accessible using a browser over the Internet. Aconsumer may visit the website hosted by website application 120 to viewone or more items for purchase. Moreover, the website may allow the userto select one or more of the item(s) for purchase and enter the selecteditem(s) to a transaction. The website may include checkout and paymentprocesses for use in completing the transaction between the user and themerchant. The website may provide shopping, checkout, and/or paymentservices through payment provider server 150 and/or using servicesoffered by payment provider server 150, for example, using salesapplication 112. Moreover, shopping and transaction informationgenerated by website application 120 and using services, processes, andfeatures provided by payment provider server 150 may be collected bypayment provider server 150 to determine a total payment volume and/orother analytical information on merchant behavior (e.g., sales, paymentprocesses used, user integration and/or satisfaction, etc.). In otherembodiments, website application 120 may instead correspond to adedicated application provided by current merchant server 110, which mayprovide data used by the application (e.g., retrievable through an APIcall of the dedicated application to website application 120).

The website provided by website application 120 may further includeassociated website data. The website data may be displayed on orotherwise retrievable from the website (e.g., through accessing storeddatabase information associated with the website). The website data maycorrespond to information provided by the merchant on the website, suchas titles, descriptions, instructions, categories, merchant information,and/or other shopping/payment information. The website data may alsoinclude information entered by consumers to or using the website. Forexample, the website data may include forum posts, user reviews,messages, user information, and/or other user entered data. The websitedata may include text, as well as text metadata (e.g., color, placement,size, font, etc.). In further embodiments, the website data may alsoinclude images, banners, logos, symbols, and other graphical elements.The website data may be retrieved by payment provider server 150, asdiscussed herein.

Sales application 112 may correspond to one or more processes to executemodules and associated specialized hardware of current merchant server110 to sell one or more items offered by a current merchant customer(not shown) of payment provider server 150 and associated with currentmerchant server 110 through a website provided by website application120. Sales application 112 may further provide checkout and paymentprocesses for a transaction to purchase the items for sale from themerchant corresponding to current merchant server 110. In this regard,sales application 112 may correspond to specialized hardware and/orsoftware of current merchant server 110 to provide shopping, checkout,and payment processes used in the website. For example, salesapplication 112 may provide item sales through an online marketplaceusing the website of the merchant. In other embodiments, salesapplication 112 may provide sale of items in a physical location, suchas a physical merchant retail location, through the website of themerchant.

Sales application 112 may include information for a price for the item,a discount for the item, a price change for the item, and/or otherincentives for items and/or with the merchant corresponding to currentmerchant server 110 (e.g., rebates, payments, etc.). Additionally, thesales data and other item data (e.g., inventory, status, condition,etc.) in the purchase information may be retrievable by the website ofcurrent merchant server 110, such as request-able through an API calland/or retrievable from database 116. The information may be basedupdated periodically or continuously, such as in real time andinformation for the item(s) for sale changes. Sales application 112 maybe used to establish a transaction once a consumer has selected one ormore items for purchase in a transaction, which may utilize checkoutprocesses provided by payment provider server 150. Once a payment amountis determined for the transaction for the item(s) to be purchased, salesapplication 112 may request payment from the consumer using processesprovided by payment provider server 150. Sales application 112 mayreceive payment processing information, such as a purchase request. Insuch embodiments, the purchase request may be processed, paymentprovided to the merchant account, and notification of payment (orfailure, for example, where there are insufficient user funds) may besent to sales application 112. The payment may be made by paymentprovider server 150 on behalf of the user associated with currentmerchant server 110. Sales application 112 may then receive the resultsof the transaction processing, and complete the transaction with theuser, for example, by providing the user the items for the transactionor declining the transaction where the user is not authenticated or thetransaction is not authorized (e.g., insufficient funds).

Current merchant server 110 includes other applications 114 as may bedesired in particular embodiments to provide features to currentmerchant server 110. For example, other applications 114 may includesecurity applications for implementing server-side security features,programmatic client applications for interfacing with appropriateapplication programming interfaces (APIs) over network 180, or othertypes of applications. Other applications 114 may also include email,texting, voice and IM applications that allow a merchant to send andreceive emails, calls, texts, and other notifications through network180. In various embodiments, other applications 114 may includefinancial applications, such as banking, online payments, moneytransfer, or other applications associated with payment provider server150. Other applications 114 may contain software programs, executable bya processor, including a graphical user interface (GUI) for a website orapplication configured to provide an interface to consumers.

Current merchant server 110 may further include database 116 which mayinclude, for example, identifiers such as operating system registryentries, identifiers associated with hardware of current merchant server110, or other appropriate identifiers, such as identifiers used forpayment/merchant/server authentication or identification. Identifiers indatabase 116 may be used by a payment/credit provider, such as paymentprovider server 150, to associate current merchant server 110 with aparticular account maintained by the payment/credit provider. Item,sales, and/or benefit information for items sold by the merchantassociated with current merchant server 110 may be stored to database116. Database 116 may further include website data and information,including term information for terms used on or with a website forcurrent merchant server 110.

Current merchant server 110 includes at least one communication module118 adapted to communicate with payment provider server 150. In variousembodiments, communication module 118 may include a DSL (e.g., DigitalSubscriber Line) modem, a PSTN (Public Switched Telephone Network)modem, an Ethernet device, a broadband device, a satellite device and/orvarious other types of wired and/or wireless network communicationdevices including microwave, radio frequency, infrared, Bluetooth, andnear field communication devices.

Prospective merchant server 130 may be implemented using any appropriatehardware and software configured for wired and/or wireless communicationwith communication device 130 and/or payment provider server 150.Prospective merchant server 130 may correspond to a device, server, orcloud computing architecture to provide sales of items, for example,through a physical merchant location and/or an online marketplaceaccessible over a network connection that has a corresponding physicallocation for retrieval of one or more sold items. Prospective merchantserver 130 may further be used to process payments for items, provideincentives for purchase of items, advertise items, and/or allow releaseof items to current merchant server 110. Although a merchant device isshown, the merchant device may be managed or controlled by any suitableprocessing device. Although only one merchant device is shown, aplurality of merchant devices may function similarly.

Prospective merchant server 130 of FIG. 1 contains a website application140, a sales application 132, other applications 134, a database 136,and a communication module 138. Website application 140, salesapplication 132 and other applications 134 may correspond to processes,procedures, and/or applications executable by a hardware processor, forexample, a software program. In other embodiments, prospective merchantserver 130 may include additional or different modules havingspecialized hardware and/or software as required.

Website application 140 may correspond to one or more processes toexecute modules and associated specialized hardware of prospectivemerchant server 130 to provide a website for a merchant, which mayinclude one or more webpages accessible over a network by a consumer inorder to shop with the merchant. In this regard, website application 140may correspond to specialized hardware and/or software of prospectivemerchant server 130 that provide a website and corresponding interfaces.The website may correspond to a merchant marketplace for a prospectivemerchant associated with prospective merchant server 130, or other salesinterface. Website application 140 may provide the website over anetwork, for example, accessible using a browser over the Internet. Aconsumer may visit the website hosted by website application 140 to viewone or more items for purchase. Moreover, the website may allow the userto select one or more of the item(s) for purchase and enter the selecteditem(s) to a transaction. The website may include checkout and paymentprocesses for use in completing the transaction between the user and themerchant. In various embodiments, the website may correspond to aprospective merchant that does not implement any services or featuresassociated with payment provider server 150. However, the website mayalso provide shopping, checkout, and/or payment services through paymentprovider server 150 and/or using services offered by payment providerserver 150. In this regard, payment provider server 150 may furtherprocess the website and associated website term data to determineadditional services and analytics to offer the prospective merchant foruse in the website. Moreover, shopping and transaction informationgenerated by website application 140 may be collected by paymentprovider server 150 to determine a total payment volume and/or otheranalytical information on merchant behavior (e.g., sales, paymentprocesses used, user integration and/or satisfaction, etc.) of theprospective merchant. In other embodiments, website application 140 mayinstead correspond to a dedicated application provided by prospectivemerchant server 130, which may provide data used by the application(e.g., retrievable through an API call of the dedicated application towebsite application 140).

The website provided by website application 140 may further includeassociated website data. The website data may be displayed on orotherwise retrievable from the website (e.g., through accessing storeddatabase information associated with the website). The website data maycorrespond to information provided by the merchant on the website, suchas titles, descriptions, instructions, categories, merchant information,and/or other shopping/payment information. The website data may alsoinclude information entered by consumers to or using the website. Forexample, the website data may include forum posts, user reviews,messages, user information, and/or other user entered data. The websitedata may include text, as well as text metadata (e.g., color, placement,size, font, etc.). In further embodiments, the website data may alsoinclude images, banners, logos, symbols, and other graphical elements.The website data may be retrieved by payment provider server 150, asdiscussed herein.

Sales application 112 may correspond to one or more processes to executemodules and associated specialized hardware of prospective merchantserver 130 to sell one or more items offered by a prospective merchantcustomer (not shown) of payment provider server 150 and associated withprospective merchant server 130 through a website provided by websiteapplication 140. Sales application 112 may further provide checkout andpayment processes for a transaction to purchase the items for sale fromthe merchant corresponding to prospective merchant server 130. In thisregard, sales application 112 may correspond to specialized hardwareand/or software of prospective merchant server 130 to provide shopping,checkout, and payment processes used in the website. For example, salesapplication 112 may provide item sales through an online marketplaceusing the website of the merchant. In other embodiments, salesapplication 112 may provide sale of items in a physical location, suchas a physical merchant retail location, through the website of themerchant.

Sales application 112 may include information for a price for the item,a discount for the item, a price change for the item, and/or otherincentives for items and/or with the merchant corresponding toprospective merchant server 130 (e.g., rebates, payments, etc.).Additionally, the sales data and other item data (e.g., inventory,status, condition, etc.) in the purchase information may be retrievableby the website of prospective merchant server 130, such as request-ablethrough an API call and/or retrievable from database 116. Theinformation may be based updated periodically or continuously, such asin real time and information for the item(s) for sale changes. Salesapplication 112 may be used to establish a transaction once a consumerhas selected one or more items for purchase in a transaction. Once apayment amount is determined for the transaction for the item(s) to bepurchased, sales application 112 may request payment from the consumer.Sales application 112 may receive payment processing information, suchas a purchase request. In such embodiments, the purchase request may beprocessed, payment provided to the merchant account, and notification ofpayment (or failure, for example, where there are insufficient userfunds) may be sent to sales application 112. Sales application 112 maythen receive the results of the transaction processing, and complete thetransaction with the user, for example, by providing the user the itemsfor the transaction or declining the transaction where the user is notauthenticated or the transaction is not authorized (e.g., insufficientfunds). In various embodiments, some or all of the aforementionedfeatures may be provided by payment provider server 150 or may beprocessed using other processes. In this regard, payment provider server150 may attempt to advertise additional services and features to theprospective merchant corresponding to prospective merchant server 130.

Prospective merchant server 130 includes other applications 134 as maybe desired in particular embodiments to provide features to prospectivemerchant server 130. For example, other applications 134 may includesecurity applications for implementing server-side security features,programmatic client applications for interfacing with appropriateapplication programming interfaces (APIs) over network 180, or othertypes of applications. Other applications 134 may also include email,texting, voice and IM applications that allow a merchant to send andreceive emails, calls, texts, and other notifications through network180. In various embodiments, other applications 134 may includefinancial applications, such as banking, online payments, moneytransfer, or other applications associated with payment provider server150. Other applications 134 may contain software programs, executable bya processor, including a graphical user interface (GUI) configured toprovide an interface of a website or application to consumers.

Prospective merchant server 130 may further include database 136 whichmay include, for example, identifiers such as operating system registryentries, identifiers associated with hardware of prospective merchantserver 130, or other appropriate identifiers, such as identifiers usedfor payment/user/device authentication or identification. Identifiers indatabase 136 may be used by a payment/credit provider, such as paymentprovider server 150, to associate prospective merchant server 130 with aparticular account maintained by the payment/credit provider. Item,sales, and/or benefit information for items sold by the merchantassociated with prospective merchant server 130 may be stored todatabase 136. Database 136 may further include website information andother data, such as website term data for terms used on or with awebsite.

Prospective merchant server 130 includes at least one communicationmodule 138 adapted to communicate with payment provider server 150. Invarious embodiments, communication module 138 may include a DSL (e.g.,Digital Subscriber Line) modem, a PSTN (Public Switched TelephoneNetwork) modem, an Ethernet device, a broadband device, a satellitedevice and/or various other types of wired and/or wireless networkcommunication devices including microwave, radio frequency, infrared,Bluetooth, and near field communication devices.

Payment provider server 150 may be maintained, for example, by an onlineservice provider, which may provide connection services on behalf ofusers. In this regard, payment provider server 150 includes one or moreprocessing applications which may be configured to interact withcommunication device 130, prospective merchant server 130, and/oranother device/server to facilitate payments and payment processing. Inone example, payment provider server 150 may be provided by PAYPAL, Inc.of San Jose, Calif., USA. However, in other embodiments, paymentprovider server 150 may be maintained by or include another type ofservice provider, which may provide connection services to a pluralityof users.

Payment provider server 150 of FIG. 1 includes a website processingapplication 160, a merchant advertisement application 170, a transactionprocessing application 152, other applications 154, a database 156, anda network interface component 158. User connection application 170,transaction processing application 152, and other applications 154 maycorrespond to executable processes, procedures, and/or applications withassociated hardware. In other embodiments, payment provider server 150may include additional or different modules having specialized hardwareand/or software as required.

Website processing application 160 may correspond to one or moreprocesses to execute software modules and associated specializedhardware of payment provider server 150 to retrieve website data for aprospective merchant customer of payment provider server 150 and processthe website data using website term information stored to database 156.In this regard, website processing application 160 may correspond tospecialized hardware and/or software to determine that prospectivemerchant server 130 corresponds to a prospective merchant customer ofpayment provider server 150 that may wish to implement one or moreprocesses, services, and/or features of payment provider server 150 intoa website of the prospective merchant. For example, website processingapplication 160 may receive a request to process the website data forthe website of prospective merchant server 130. In other embodiments,website processing application 160 may instead comb or scrape availablewebpages and/or perform webpage searches using terms from the websiteterm information to determine the webpage for prospective merchantwebsite 130 is a website of a prospective merchant.

Prior to determination of the website of prospective merchant server 130is a website for a prospective merchant customer, website processingapplication 160 may determine the website term information for pastand/or current merchant customers of payment provider server 150. Thewebsite term information may include processed website data havingtexts, graphics, and/or images (e.g., terms) used on the past and/orcurrent merchant customers of payment provider server 150 that utilizeone or more service of payment provider server 150. Thus, the websitedata may correspond to a website of the merchant having text, images,and other data represented on the website of the merchant. In furtherembodiments, a dedicated application having one or more applicationinterfaces may also be used. Website processing application 160 may scanthe websites (and/or dedicated applications) of the past and/or currentmerchant customers (e.g., the website hosted by website application 120of current merchant server 110) or pull the website data from a database(e.g., database 116 of current merchant server 110). Once pulled,website processing application 160 may parse the data to determine textand other terms included on the merchant websites (and/or dedicatedapplications). The terms may include terms used by the merchant on theirwebsite (e.g., information displayed on a front webpage and/orsub-webpages, sales, payment, checkout, merchant, and/or consumerinformation). The terms may also include information provided byconsumers on the website, including product reviews, forum postings,messages, service requests, and other types of information submitted byusers to the website. The terms may correspond to a single text element,such as a character or word, as well as multiple text elements (e.g., asentence). The terms may also include information on usage, such asnearby terms and/or accompanying terms, placement in a sentence orparagraph, or other term information. In various embodiments, the termsmay further include visual images appearing on the merchant website, aswell as logos and other graphics. Such images may be processed usingimage processing techniques to determine the content of the image, ormay be matched to pre-existing images, such as similar logos, banners,and graphics used by merchants, payment providers, customer users,and/or service providers.

Thus, website processing application 160 may determine the website terminformation having a dictionary of terms used on webpages of existingand/or past merchant customers of payment provider server 150 (e.g.,merchants that utilize services, processes, and/or features provided bypayment provider server 150). The dictionary may include one or moreterms, as well as analytics for each term, such as rankings or numbersof how often terms appear on the websites, on what percentage of thewebsites that the terms appear, where they appear on the websites,and/or other analytic associated with the appearance of the terms on thewebsites. In this regard, the terms may provide an indication of theterms importance or prevalence of appearance on the merchant websites.For example, a term appearing on a high number/percentage of websitesand/or having a large number of appearances may be indicated asimportant to determining similar merchants that may wish to use servicesprovided by payment provider server 150. Conversely, a term appearingonly a few times and/or on a low percentage of the websites may not beindicative of a potential merchant customer of the payment provider.Thus, the number of times the term appears on a single website andacross a plurality of merchants' websites may be important. In otherembodiments, other data for the term may also be used to score or rankthe term, or may provide other indications of how well the term performsin predicting behavior of other prospective merchant clients based ontheir website data. For example, the placement of a term on the websitemay be important (e.g., on the front webpage of a website or a sub-pageof the website), a size or font of the term (e.g., terms with large sizeor prominent fonts so as to be visible), a color of the term, and/or aratio of the term to other terms and elements on the website (e.g., ifthe term appears in a high ratio or where the term may appear in a lowratio due to a high volume of text and other terms presented on thewebsite).

Website processing application 160 may further determine a total paymentvolume by the merchant, which may correspond to a number, amount, and/orfrequency of payments to the merchant by the merchant's consumer base.The total payment volume may indicate the merchant's performance andwell as the payment services of payment provider server 150 that wouldbe ideal for the merchant. Website processing application 160 may alsodetermine the payment processing features and services that eachmerchant uses. The aforementioned data may be paired with the websiteterm information to associate total payment volumes and/or paymentprocesses with various website terms.

In various embodiments, website processing application 160 may updatethe dictionary of terms and their corresponding information (e.g.,occurrence, importance, score, associated total payment volume, and/orassociated payment processes) based on new information retrievable fromeach merchant website. Website processing application 160 may also addnew and/or prospective merchant websites into the dictionary,particularly if the prospective merchant becomes a customer of thepayment provider (e.g., purchases and/or implements the paymentprovider's processes). The dictionary may also use historicalinformation for the websites and/or with the payment provider, which mayinclude when merchants became a customer of the payment provider,updates to websites and added/removed content, and other data. The termusage may be country and/or language specific, and may be weigheddifferently based on the country/language.

Website processing application 160 may pull or retrieve the website datafor the website, for example, from a database associated with thewebsite (e.g., database 136 of prospective merchant server 130). Inother embodiment, website processing application 160 may instead scrapethe data from displayed or retrievable data on the website, for example,from one or more webpages constituting the website (e.g., the websitehosted by website application 140 of prospective merchant server 130).Once determined, website processing application 160 may process thewebsite data for prospective merchant server 130 using the website terminformation to determine services and/or analytics to offer toprospective merchant server 130. For example, based on one or morematching terms, website processing application 160 may determine whetherprospective merchant server 130 would be a candidate to utilize one ormore services of payment provider server 150, as well as what servicesprospective merchant server 130 may wish to utilize. Website processingapplication 160 may also determine a predicted total payment volumebased on the appearance of one or more terms and their associatedmerchants. In various embodiments, a score may also be determined forthe website, which may correspond to number of appearance, number ofterms, strength of each term, and/or the merchants matching the termstotal payment volume and/or used payment services. Other statistics,such as the number of appearances, the ratio appearances to other textand terms, the size, the color, the placement, and/or other informationfor the term may all be used in the score. In various embodiments, adedicated application of prospective merchant server 130 may instead beprocessed against a dictionary of terms for dedicated applications ofpast and current merchant customers of payment provider server 150.Thus, the dictionary may be platform specific and rely on terms usedwithin the application or website, or may be platform agnostic and relyinstead on term usage.

Merchant advertisement application 170 may correspond to one or moreprocesses to execute software modules and associated specializedhardware of payment provider server 150 to generate advertisementsand/or message to prospective merchants determined using websiteprocessing application 160 of services and analytics, and communicatethe messages to the prospective merchants. In this regard, merchantadvertisement application 170 may correspond to specialized hardwareand/or software to access prospective merchant information determinedusing website processing application 160. The prospective merchantinformation may include prospective merchants (e.g., the merchantcorresponding to prospective merchant server 130) as well as the totalpayment volume, potential payment services, and/or other information forthe merchant. Merchant advertisement application 170 may generate anadvertisement for to prospective merchant server 130 based on theprospective merchant information determined using the merchant's websitedata and the website term information. The advertisement may generallyadvertise the services of payment provider server 150 to prospectivemerchant server 130. In other embodiments, the advertisement may betargeted to the services determined to be useful to prospective merchantserver 130. Moreover, the advertisements may also include messages ofanalytics or other data for the prospective merchant using the websiteterm information. In yet other embodiments, an advertisement responsemodel may be provided to the merchant advertisement application 170, andthe merchant advertisement application 170 may be configured to performa model enhancement process to improve the advertisement response model.The model enhancement process is described in more detail below withreference to FIG. 5 and FIG. 6.

Transaction processing application 152 may correspond to one or moreprocesses to execute software modules and associated specializedhardware of payment provider server 150 to provide payment services tomerchant and users, for example though a user's payment account and/orpayment instruments. In this regard, transaction processing application152 may correspond to specialized hardware and/or software to providepayment services, which may be implemented into one or more merchantwebsites and/or dedicated applications of a merchant. The paymentservices may allow for a payment to the merchant by a user through apayment instrument, including a credit/debit card, banking account,payment account with payment provider server 150, and/or other financialinstrument. In order to establish a payment account for a merchantand/or user to send and receive payments, transaction processingapplication 152 may receive information requesting establishment of thepayment account. The information may include user personal and/orfinancial information. Additionally, the information may include alogin, account name, password, PIN, or other account creationinformation. The merchant/user may provide a name, address, socialsecurity number, or other personal information necessary to establishthe account and/or effectuate payments through the account. Transactionprocessing application 152 may further allow the merchant/user toservice and maintain the payment account, for example, by adding andremoving payment instruments. Transaction processing application 152 maybe used to provide a payment for an item to a merchant, such as currentmerchant server 110 and/or prospective merchant server 130. Transactionprocessing application 152 may debit an account of the userautomatically and provide the payment to an account of the merchant.Transaction processing application 152 may also be used to providetransaction histories for processed transactions.

In various embodiments, payment provider server 150 includes otherapplications 154 as may be desired in particular embodiments to providefeatures to payment provider server 134. For example, other applications154 may include security applications for implementing server-sidesecurity features, programmatic client applications for interfacing withappropriate application programming interfaces (APIs) over network 180,or other types of applications. Other applications 154 may containsoftware programs, executable by a processor, including a graphical userinterface (GUI), configured to provide an interface to the user whenaccessing payment provider server 150, where the user or other users mayinteract with the GUI to more easily view and communicate information.In various embodiments, other applications 154 may include connectionand/or communication applications, which may be utilized to communicateinformation to over network 180.

Additionally, payment provider server 150 includes database 156. Aspreviously discussed, the user and/or the merchant corresponding totelecommunication carrier server 170 may establish one or more digitalwallets and/or payment accounts with payment provider server 150.Digital wallets and/or payment accounts in database 156 may include userinformation, such as name, address, birthdate, paymentinstruments/funding sources, additional user financial information, userpreferences, and/or other desired user data. Users may link to theirrespective digital wallets and/or payment accounts through an account,user, merchant, and/or device identifier. Thus, when an identifier istransmitted to payment provider server 150, e.g. from communicationdevice 130, one or more digital wallets and/or payment accountsbelonging to the users may be found. Additionally, database 156 maystore data for various websites, as well as process website terminformation and information on prospective merchant customers forpayment provider server 150.

In various embodiments, payment provider server 150 includes at leastone network interface component 158 adapted to communicate currentmerchant server 110 and/or prospective merchant server 130 over network180. In various embodiments, network interface component 158 maycomprise a DSL (e.g., Digital Subscriber Line) modem, a PSTN (PublicSwitched Telephone Network) modem, an Ethernet device, a broadbanddevice, a satellite device and/or various other types of wired and/orwireless network communication devices including microwave, radiofrequency (RF), and infrared (IR) communication devices.

Network 180 may be implemented as a single network or a combination ofmultiple networks. For example, in various embodiments, network 180 mayinclude the Internet or one or more intranets, landline networks,wireless networks, and/or other appropriate types of networks. Thus,network 180 may correspond to small scale communication networks, suchas a private or local area network, or a larger scale network, such as awide area network or the Internet, accessible by the various componentsof system 100.

FIG. 2 is a representation of two exemplary merchant website interfacesfor use in determining a dictionary of website terms and providingtargeted advertisement of payment services, according to an embodiment.Environment 200 includes a merchant website A 1000 corresponding to acurrent merchant customer website provided by website application 120executed by current merchant server 110 in environment 100 of FIG. 1.Moreover, environment 200 includes a merchant website B 1100corresponding to a prospective merchant customer website provided bywebsite application 140 executed by prospective merchant server 130 inenvironment 100 of FIG. 1.

Merchant website A 1000 may correspond to a website of a currentmerchant customer of a payment provider that utilizes various shopping,checkout, and/or payment services and features provided by the paymentprovider. In this regard, merchant website A 1000 includes a webpage A1002 having data that may include terms used on merchant website A 1000.For example, webpage A 1002 includes a sales portal A 1004 for sales ofitems. Sales portal A 1004 may include a term A 1006, such as text,logos, banners, images, and/or graphics. Term A 1006 may be included inthe website data for merchant website A 1000 as a term used on merchantwebsite A 1000. Sales portal A 1004 may also include an item A 1008having a term B 1010, a term C 1012, and a term D 1014. Sales portal A1004 may also include a checkout process A 1016.

Webpage A 1002 may further include payment processes for use with salesportal A 1004. For example, a customer of merchant website A 1000 mayutilize payment process A 1018 and/or payment process B 1020. Paymentprocess A 1018 and payment process B 1020 may correspond to paymentprocesses offered by the payment provider. Moreover, in certainembodiments, processes and features of sales portal A 1004 and/orcheckout process A 1016 may also be offered by the payment provider. Thepayment provider may further process data from merchant A information1022, such as term E 1024 and term F 1026 in various embodiments.

Merchant website B 1100 may correspond to a web site of a prospectivemerchant, which may be a candidate to utilize one or more of theprocesses or services provided by the payment provider. Thus, thepayment provider may process data available from webpage B 1102. Forexample, webpage B 1102 may include information for sales portal B 1104.Thus, the payment provider may determine that webpage B includes term G1106, as well as the terms for item A 1008 (e.g., term H 1108, term C1012, and term D 1014). As term C 1012 and term D 1014 match betweenwebpage A 1002 of the current merchant customer of the payment providerand webpage B 1102 of the prospective merchant customer of the paymentprovider, the payment provider may determine that merchant website B1100 may be a prospective customer, may wish to utilize one or moreprocesses or services utilized by merchant website A 1000, and/or mayprovide analytics on similar current merchant customers. For example,although webpage B 1102 uses checkout process B 1110, webpage B 1102 maybetter utilize checkout process A 1016. Similarly, although paymentprocess C 1112 is used on webpage B 1102, payment process A 1018 ofwebpage A 1002 may be better or ideal.

In similar fashion, additional matching information may also be used toprovide similarities between merchant website A 1000 and merchantwebsite B 1100. For example, payment process B 1020 is shared betweenwebpage A 1002 and webpage B 1102. In addition, merchant B informationfor merchant website B 1100 has a new term H 1116, but shares term F1026 with merchant A information for merchant website A 1000. Thus,additional information and terms may be shared between websites.

FIG. 3 is an exemplary system environment having merchant websiteservers with data necessary for determining a dictionary of websiteterms and providing targeted advertisement of payment services,according to an embodiment. FIG. 3 includes current merchant server 110,a prospective merchant server 130, and a payment provider server 150 alldiscussed in reference to environment 100 of FIG. 1.

Current merchant server 110 executes website application 120corresponding generally to the specialized hardware and/or softwaremodules and processes described in reference to FIG. 1. In this regard,website application 120 includes information for a website hosted by acurrent merchant customer of payment provider server 150. For example,website information 2000 may be utilized with the website. Websiteinformation 2000 may include data that may be extracted or determined bypayment provider server 150. Thus, website information 2000 includeswebsite terms 2002. Website information 2000 may also include merchantinformation 2004 and sales information 2006, such as checkout processes20008 and payment processes 2010.

Similarly, prospective merchant server 130 executes website application140 corresponding generally to the specialized hardware and/or softwaremodules and processes described in reference to FIG. 1. In this regard,website application 140 includes information for a website hosted by aprospective merchant customer of payment provider server 150. Websiteinformation 2100 may be processed by payment provider server 150 toprovide services and analytics to the prospective merchant customer.Website information similarly includes website terms 2102 used to matchto website terms 2002 of current merchant server 110. Additionally,merchant information 2104 and sales information 2106 having checkoutprocesses 2108 and payment processes 2110 may be matched to merchantinformation 2004 and sales information 2006 of current merchant server110. In various embodiments, payment provider server 150 may offerchanges and upgrades to sales information 2106, such as new checkoutand/or payment processes.

Payment provider server executes website processing application 160 andmerchant advertisement application 170 corresponding generally to thespecialized hardware and/or software modules and processes described inreference to FIG. 1. In this regard, website processing application 160may process website information 2000 and website information 2100. Forexample, website information 2100 may be included in retrieved websiteinformation 2200, which may also include website terms 2102 for websiteinformation 2100. Website term information 2202 includes informationprocesses from a plurality of current merchant customer of paymentprovider server 150, such on the websites of the current merchantcustomers. Thus, website term information 2202 includes websiteinformation 2000 in stored websites and terms 2204. Stored websites andterms 2204 further includes associated services 2206 for the servicesprovided by the websites in stores websites and terms 2204. Using theaforementioned information, website processing application 160 maydetermine matching website information 2208, which may include matchingterms 2210.

Merchant advertisement application 170 may be used to provideadvertisements and/or messages based on matching website information2208. For example, merchant advertisement application 170 includesdetermined advertisements 2300, such as advertisement A 2302.Advertisement A 2302 may include matching terms 2210. Moreover,advertisement A 2302 includes information offered to a prospectivemerchant, such as offered services 2304 and/or offered information 2306(e.g., merchant analytics).

FIG. 4 is a flowchart of an exemplary process for predicting merchantbehavior using merchant website terms, according to an embodiment. Notethat one or more steps, processes, and methods described herein may beomitted, performed in a different sequence, or combined as desired orappropriate.

At step 402, merchant website data for a website of a merchant isretrieved, by a payment provider system that comprises one or morehardware processors coupled to a non-transitory memory, merchant websitedata for a website of a merchant, wherein the merchant website datacomprises merchant website terms on the website of the merchant. Themerchant website data may be for a plurality of merchant webpagesconstituting the website of the merchant. The merchant website data maycomprise text used by the merchant on the plurality of merchantwebpages. The merchant website data may also include text on theplurality of merchant webpages by at least one customer of the merchantin a forum, website postings, messages, and requests on the plurality ofmerchant webpages. The merchant website data may further comprise a textsize, a color, a font, a sequence of terms, a ratio of a term to otherterms, and a type of merchant for the merchant. The website of themerchant may be one of a mobile website type, a dedicated applicationwebsite type, and a traditional website type, wherein the website terminformation is particular to one of the mobile website type, thededicated application website type, and the traditional website type.

Website term information is accessed, from the non-transitory memory,wherein the website term information comprises a plurality of termsappearing on a plurality of merchant websites for a plurality ofmerchants, at step 404. For example, the plurality of merchants maycomprise previous customers and current customers of the paymentprovider system utilizing at least one service provided by the paymentprovider system. Thus, the merchant may comprise a prospective customerof the payment provider system. In various embodiments, the website terminformation is determined using the plurality of merchant websites forthe plurality of merchants, wherein the website term information isdetermined based on occurrences of the plurality of terms on theplurality of merchant websites and current payment processing servicesutilized by each of the plurality of merchants.

Moreover, each of the plurality of terms in the website term informationmay further be associated with a weighted score based on an importancefor the each of the plurality of terms being found on the merchantwebsite when determining the advertisement. For example, the weightedscore for the each of the plurality of terms may be determined based onpayment processing services used by the plurality of merchants and theplurality of terms occurring on each of the plurality of merchantwebsites. At least one of the merchant website terms on the website ofthe merchant is determined to match at least one of the plurality ofterms in the website term information, at step 406. For example, themerchant may be linked to at least one of the plurality of merchantsbased on the merchant website terms and the plurality of terms.

At step 408, an advertisement for a payment processing service providedby the payment provider system is determined based on the at least oneof the merchant website terms matching the at least one of the pluralityof terms. In various embodiments, the merchant website data includes anumber of occurrences for each of the merchant website terms on thewebsite of the merchant, wherein the advertisement is further determinedbased on the number of occurrences. In further embodiments, a totalscore for the website of the merchant may be determined using the scoreassociated with each of the plurality of terms, wherein at least one ofthe advertisement and the payment processing service is furtherdetermined using the total score. For example, the total score may beused to determine a prediction comprising at least one of a merchantsize, payment sizes for payments to the merchant, payment volume for thepayments to the merchant, and a customer type. Thus, at least one of theadvertisement and the payment processing service is determined using theprediction.

The advertisement is communicated to the merchant, at step 410. Thepayment processing service in the advertisement may comprise a paymentprocess for merchant customers to provide payments to the merchant, awebsite payment flow for the website of the merchant, a payment accountservice with the payment provider system, and a checkout process on thewebsite of the merchant. The payment processing service may beimplemented in the website of the merchant if the merchant accepts theadvertisement. In various embodiments, the process may further includeupdating the website term information with the merchant website terms ifthe merchant accepts the payment processing service, wherein themerchant website terms are associated with the payment processingservice. Thus, at least one new merchant website to retrieve additionalmerchant website data from may be determined based on the updatedwebsite term information.

FIG. 5 is a block diagram of a networked system 200 that includes thepayment provider server 150, a user computer 502 and a prospective usercomputer 504. System 200 may comprise or implement a plurality ofdevices, servers, and/or software components that operate to performvarious methodologies in accordance with the described embodiments.Exemplary devices and servers may include device, stand-alone, andenterprise-class servers, operating an OS such as a MICROSOFT® OS, aUNIX® OS, a LINUX® OS, or other suitable device and/or server based OS.It can be appreciated that the devices and/or servers illustrated inFIG. 2 may be deployed in other ways and that the operations performedand/or the services provided by such devices and/or servers may becombined or separated for a given embodiment and may be performed by agreater number or fewer number of devices and/or servers. One or moredevices and/or servers may be operated and/or maintained by the same ordifferent entities.

The user computer 502 and the prospective user computer 504 may eachinclude one or more processors, memories, and other appropriatecomponents for executing instructions such as program code and/or datastored on one or more computer readable mediums to implement the variousapplications, data, and steps described herein. For example, suchinstructions may be stored in one or more computer readable media suchas memories or data storage devices internal and/or external to variouscomponents of system 200, and/or accessible over network 180. In certainembodiments, the user computer 502 may include the current merchantserver 110, and the prospective user computer 504 may include theprospect merchant server 130. In other embodiments, the user computer502 may include various other computers and electronic devicesassociated with users of the payment provider. Similarly, theprospective computer 504 may include various other computers andelectronic devices associated with prospective users of the paymentprovider. As used herein, a “prospective user” may refer to an entity(e.g., a person, consumer, merchant, and/or the like) that is not yet auser of the payment provider.

As illustrated in FIG. 5, the user computer 502, the prospective usercomputer 504, and the payment provider server 150 may be incommunication via the network 180. Further, the payment provider server150 may include an advertisement modeling application 510. As describedmore fully below, the advertisement modeling application 510 may beprovided an advertisement response model. The advertisement responsemodel may include, but is not limited to classification algorithms suchas AdaBoost classifier, logistic regression, support vector machines,gradient boosting classifier, Isolation Forest, linear discriminantanalysis, neural networks (e.g., multi-layer perceptron), decision treesclassifier, Random Forest classifier, naïve Bayesian classifier, andBagging classifier. The advertisement modeling application 510 mayimprove the advertisement response model using a model enhancementprocess. The advertisement response model may facilitate a selection ofwhich users of the user computers 502 and which prospective users of theprospective user computers 504 to target with respect to an advertisingcampaign. In implementations where certain selected users and/orprospective users correspond to merchant and/or prospective merchants,the advertisement modeling application 510 may communicate with themerchant advertisement application 170 to transmit advertisements to thecorresponding current merchant server(s) 110 and/or prospective merchantservers 130.

FIG. 6 depicts a flow chart of a method 600 for performing a modelenhancement process that improves an advertisement response model inaccordance with one or more particular embodiments. As previouslydiscussed, the advertisement response model may be configured to predictthe responses of units included in a target data set to a particularadvertising campaign. Further, it will be appreciated that while themethod 600 may be described as being performed by the merchantadvertisement application 170, the method 600 may also be performed byother applications 154 of the payment provider server 150 or by anotherapplication included in another computing device, or a combinationthereof. Note that one or more steps, processes, and methods describedherein may be omitted, performed in a different sequence, or combined asdesired or appropriate.

At step 602, a training data set may be generated. In certainimplementations, the training data set may have been previouslygenerated, such as by a data gather device (not illustrated), and may beprovided to the advertisement modeling application 510. In otherembodiments, the training data set may be generated, such as by theadvertisement modeling application 510. For example, the advertisementmodeling application 510 may generate the training data set based on arun of the advertising campaign with respect to a random sampling of aparticular data set. As such, the responses of each unit of the randomsampling to the advertising campaign may be stored in the training dataset. It will be appreciated that other methods of generating thetraining data set are also contemplated. Furthermore, in someembodiments, the target data set, the training data set, and theparticular data set may be different from each other.

At step 604, the advertisement modeling application 510 may execute theadvertisement response model using the training data set. In otherwords, the advertisement response model may be trained by the trainingdata set. At step 606, the advertisement modeling application 510 maycalculate a accuracy value associated with the advertisement responsemodel. The advertisement modeling application 510 may determine whetherthe calculated accuracy value meets (e.g., reaches or exceeds) aaccuracy value threshold. The accuracy value may measure an ability ofthe advertisement response model to accurately predict how particularusers or prospective users would respond to the advertising campaign. Incertain embodiments, a higher accuracy value may indicate a greateraccuracy in predictions by the advertisement response model. Accordingto a particular embodiment, the accuracy value may be calculated usingpredictive entropy. However, it will be appreciated that various otheralgorithms may also be used calculate the accuracy value. For example,in certain embodiments, the accuracy value may be calculated as part ofexecuting the advertisement response model.

If the accuracy value does not meet the score threshold, the method 600may proceed to step 608. At step 608, the advertisement modelingapplication 510 may execute the advertisement response model using thetarget data set. In certain implementations, executing the advertisementresponse model at step 608 may include testing the advertisementresponse model using the target data set. Further, at step 610, theadvertisement modeling application 510 may determine certainty scoresassociated with the target data set. According to a particularembodiment, the advertising modeling application 510 may determine arespective certainty score for each unit in the target data set.Further, the certainty scores may be calculated using variousalgorithms, such as Stochastic Gradient Descent. and may be calculatedas part of executing the advertisement response model.

At step 612, based on the certain scores, the advertisement modelingapplication 510 may determine a subset of the target data set for whichto run the advertising campaign. For example, the advertisement modelingapplication 510 may rank the units in the target data set according totheir respective certainty scores. As such, the subset of the targetdata set may include a predetermined number of lowest ranked units fromthe target data set. In other words, the remaining units in the targetdata set that are not in the subset may each have a correspondingcertainty score that is greater than each of the certainty scorescorresponding to the units included in the subset. In other examples,each unit included in the subset of the target data set may havecorresponding certainty scores below a certainty score threshold to beincluded in the subset.

In certain implementations, the certainty score for a particular unitmay indicate a degree of certainty corresponding to the advertisementresponse model's prediction as to how the particular unit would respondto the advertising campaign. For instance, a first unit having a firstcertainty score that is greater than a second certainty score of asecond unit may indicate that the advertisement response model is morecertain about the first unit's predicted response to the advertisingcampaign than the second unit's predicted response to the advertisingcampaign.

Moreover, one or more units of the subset of the target data may beassociated with certain characteristics. For example, the subset of thetarget data may be associated with particular merchant characteristicsthat include particular website terms. Thus, using this example, thesubset may indicate the advertisement response model is relativelyuncertain how merchants (who use the particular website terms on theirwebsites) would respond to the advertising campaign. It will beappreciated that other characteristics may also be associated with theunits of the subset, including but not limited to characteristics thatrelate to demographics, geographic location, buying habits, otherwebsite data, products and services being offered, and/or the like.

According to a particular embodiment, subsequent to determining thesubset of the target data set, the units included in the subset may beremoved from the target data set. Further, a run of the advertisingcampaign may be conducted with respect to the units included in thesubset. For example, one or more advertisements may be transmitted(e.g., by the merchant advertisement application 170 and/or theadvertisement modeling application 510) to the user computer(s) 502 andthe prospective user computer(s) 504.

At step 614, one or more responses to the run of the advertisingcampaign with respect to the subset may be received by the advertisementmodeling application 510. As a result, each unit of the subset may beassociated with a response type. For example, response types may includepositive responses and negative responses. A positive responseassociated with a particular unit may indicate that the particular unitresponded to the advertising campaign by purchasing a product or serviceoffered by the advertising campaign. In other words, the particular unitis “converted” by the advertising campaign. It will be appreciated thatother criteria for indicating a positive response are also possible,such as simply receiving an inquiry from the particular unit about theproduct or service. A negative response associated with the particularunit may indicate that the particular units did not purchase the productor service offered by the advertising campaign. It will be appreciatedthat other criteria for indicating a negative response are alsopossible, such as not receiving a response to the advertising campaignfrom the particular unit within a predetermined time frame.

At step 616, the advertisement modeling application 510 may update thetraining data set with the subset and the associated responses to theadvertising campaign. In certain embodiments, updating the training dataset may include adding the subset and the associated responses to thetraining data set. In other embodiments, updating the training data setmay include generating a new training data set that includes thetraining data set and the subset (and associated responses).

Subsequent to updating the training data set, the method 600 may proceedback to step 604. Thus, steps 604-616 may be repeated until a accuracyvalue of the advertisement response model meets or exceeds the accuracyvalue threshold in step 606. If the accuracy value meets or exceeds theaccuracy value threshold, the method 600 may proceed to step 618. Inthis manner, the training data set may be continuously updated, and theadvertisement response model may be iteratively trained and improved.

At step 618, the advertisement modeling application 510 may execute theadvertisement response model using the remaining units in the targetdata set. For instance, as described above with respect to the step 612,one or more units in the target data set may have been removed, sinceresponses to the advertising campaign by the units in the identifiedsubsets may already be known.

At step 620, based on executing the advertisement response model usingthe target data set, the advertisement modeling application 510 mayidentify which units in the target data set to target with respect tothe advertising campaign. For instance, the advertisement response modelmay indicate that one or more of the remaining units in the target dataset correspond to a positive predicted response to the advertisingcampaign. As such, the advertisement modeling application 510 may selectthe one or more of the remaining units to target with respect to theadvertising campaign.

As previously discussed, the method 600 enables iterative improvementsto the advertisement response model by updating the training data setwith targeted subsets of the target data set. For example, during eachiteration (e.g., of steps 604-616) the advertising campaign may be runagainst identified units in the target data set for which theadvertisement response model is relatively uncertain. As such, thisuncertainty is mitigated by obtaining the identified units' actualresponses to the advertising campaign. The training data set is thenupdated and improved in the following iteration using the obtainedresponses of the identified units, which in turn improves the trainingof the advertisement response model.

FIG. 7 is a block diagram of a computer system suitable for implementingone or more components in FIG. 1 and FIG. 5, according to an embodiment.In various embodiments, the communication device may comprise a personalcomputing device (e.g., smart phone, a computing tablet, a personalcomputer, laptop, a wearable computing device such as glasses or awatch, Bluetooth device, key FOB, badge, etc.) capable of communicatingwith the network. The service provider may utilize a network computingdevice (e.g., a network server) capable of communicating with thenetwork. It should be appreciated that each of the devices utilized byusers and service providers, including the current merchant server 110,the prospective merchant server 130, the payment provider sever 150, theuser computer 502, and the prospective user computer 504, may beimplemented as computer system 700 in a manner as follows.

Computer system 700 includes a bus 702 or other communication mechanismfor communicating information data, signals, and information betweenvarious components of computer system 700. Components include aninput/output (I/O) component 704 that processes a user action, such asselecting keys from a keypad/keyboard, selecting one or more buttons,image, or links, and/or moving one or more images, etc., and sends acorresponding signal to bus 702. I/O component 704 may also include anoutput component, such as a display 711 and a cursor control 713 (suchas a keyboard, keypad, mouse, etc.). An optional audio input/outputcomponent 705 may also be included to allow a user to use voice forinputting information by converting audio signals. Audio I/O component705 may allow the user to hear audio. A transceiver or network interface706 transmits and receives signals between computer system 700 and otherdevices, such as another communication device, service device, or aservice provider server via network 180. In one embodiment, thetransmission is wireless, although other transmission mediums andmethods may also be suitable. One or more processors 712, which can be amicro-controller, digital signal processor (DSP), or other processingcomponent, processes these various signals, such as for display oncomputer system 700 or transmission to other devices via a communicationlink 718. Processor(s) 712 may also control transmission of information,such as cookies or IP addresses, to other devices.

Components of computer system 700 also include a system memory component714 (e.g., RAM), a static storage component 716 (e.g., ROM), and/or adisk drive 717. Computer system 700 performs specific operations byprocessor(s) 712 and other components by executing one or more sequencesof instructions contained in system memory component 714. Logic may beencoded in a computer readable medium, which may refer to any mediumthat participates in providing instructions to processor(s) 712 forexecution. Such a medium may take many forms, including but not limitedto, non-volatile media, volatile media, and transmission media. Invarious embodiments, non-volatile media includes optical or magneticdisks, volatile media includes dynamic memory, such as system memorycomponent 714, and transmission media includes coaxial cables, copperwire, and fiber optics, including wires that comprise bus 702. In oneembodiment, the logic is encoded in non-transitory computer readablemedium. In one example, transmission media may take the form of acousticor light waves, such as those generated during radio wave, optical, andinfrared data communications.

Some common forms of computer readable media include, for example,floppy disk, flexible disk, hard disk, magnetic tape, any other magneticmedium, CD-ROM, any other optical medium, punch cards, paper tape, anyother physical medium with patterns of holes, RAM, PROM, EEPROM,FLASH-EEPROM, any other memory chip or cartridge, or any other mediumfrom which a computer is adapted to read.

In various embodiments of the present disclosure, execution ofinstruction sequences to practice the present disclosure may beperformed by computer system 500. In various other embodiments of thepresent disclosure, a plurality of computer systems 500 coupled bycommunication link 518 to the network (e.g., such as a LAN, WLAN, PTSN,and/or various other wired or wireless networks, includingtelecommunications, mobile, and cellular phone networks) may performinstruction sequences to practice the present disclosure in coordinationwith one another.

Where applicable, various embodiments provided by the present disclosuremay be implemented using hardware, software, or combinations of hardwareand software. Also, where applicable, the various hardware componentsand/or software components set forth herein may be combined intocomposite components comprising software, hardware, and/or both withoutdeparting from the spirit of the present disclosure. Where applicable,the various hardware components and/or software components set forthherein may be separated into sub-components comprising software,hardware, or both without departing from the scope of the presentdisclosure. In addition, where applicable, it is contemplated thatsoftware components may be implemented as hardware components andvice-versa.

Software, in accordance with the present disclosure, such as programcode and/or data, may be stored on one or more computer readablemediums. It is also contemplated that software identified herein may beimplemented using one or more general purpose or specific purposecomputers and/or computer systems, networked and/or otherwise. Whereapplicable, the ordering of various steps described herein may bechanged, combined into composite steps, and/or separated into sub-stepsto provide features described herein.

The foregoing disclosure is not intended to limit the present disclosureto the precise forms or particular fields of use disclosed. As such, itis contemplated that various alternate embodiments and/or modificationsto the present disclosure, whether explicitly described or impliedherein, are possible in light of the disclosure. Having thus describedembodiments of the present disclosure, persons of ordinary skill in theart will recognize that changes may be made in form and detail withoutdeparting from the scope of the present disclosure. Thus, the presentdisclosure is limited only by the claims.

What is claimed is:
 1. A method, comprising: electronically scanning, bya computer comprising one or more hardware processors, websites of aplurality of merchants; determining, by the computer, a frequency ofusage of one or more terms on the websites, a placement of the one ormore terms on the websites, or a visual appearance of the one or moreterms on the websites; determining, by the computer based on thefrequency, the placement, or the visual appearance of the one or moreterms on the websites, a dictionary of terms associated with thewebsites; correlating the terms in the dictionary of terms with types ofmerchants from the plurality of merchants or with payment services usedby plurality of merchants; retrieving website data from a website of afirst merchant from the plurality of merchants; determining, based onthe retrieved website data, whether any terms appearing on the websiteof the first merchant match the terms in the dictionary of terms;selecting, based on the determining whether any terms appearing on thewebsite of the first merchant match the terms in the dictionary ofterms, one or more payment provider services to advertise to the firstmerchant; training, by the computer, an advertisement response modelusing a training data set that indicates a respective response to anadvertising campaign for each unit in the training data set, whereinunits of the training data set correspond to a first subset of theplurality of merchants; determining, by the computer based on thetraining the advertisement response model using the training data set,that a first accuracy value corresponding to the advertisement responsemodel is less than an accuracy value threshold; determining, by thecomputer using the dictionary of terms, a target data set that isdifferent from the training data set, wherein units of the target dataset correspond to a second subset of the plurality of merchants, thesecond subset being different from the first subset; identifying, by thecomputer based on executing the advertisement response model using thetarget data set, one or more units from the target data set for which torun the advertising campaign; receiving, by the computer, one or moreresponses corresponding to a run of the advertising campaign withrespect to the identified one or more units from the target data set;updating, by the computer, the training data set based on the one ormore responses; and determining, by the computer based on training theadvertisement response model using the updated training data set,whether a second accuracy value corresponding to the advertisementresponse model is greater than the accuracy value threshold.
 2. Themethod of claim 1, further comprising: determining that the secondaccuracy value is greater than or equal to the accuracy value threshold;adjusting the target data set by removing the units included in theidentified one or more units from the target data set; and based onexecuting the advertisement response model with respect to the adjustedtarget data set, determining one or more units of the adjusted targetdata set that have a positive predicted response to the advertisingcampaign.
 3. The method of claim 2, further comprising: transmitting, bythe computer, an advertisement to one or more devices associated withthe determined one or more units of the adjusted target data set.
 4. Themethod of claim 1, further comprising: ranking each unit of the targetdata set according to a respective certainty score for each unit of thetarget data set, wherein each of the identified one or more units fromthe target data set is associated with a lower respective certaintyscore than respective certainty scores of the other units in the targetdata set.
 5. The method of claim 1, wherein each unit of the identifiedone or more units of the target data set is associated with a certaintyscore that is less than a certainty score threshold.
 6. The method ofclaim 1, wherein the second accuracy value is greater than the firstaccuracy value.
 7. The method of claim 1, further comprising removingthe identified one or more units from the target data set.
 8. Anon-transitory computer readable medium having stored thereonmachine-readable instructions executable to cause a machine to performoperations comprising: electronically scanning websites of a pluralityof merchants; determining a frequency of usage of one or more terms onthe websites, a placement of the one or more terms on the websites, or avisual appearance of the one or more terms on the websites; determining,based on the frequency, the placement, or the visual appearance of theone or more terms on the websites, a dictionary of terms associated withthe websites; correlating the terms in the dictionary of terms withtypes of merchants from the plurality of merchants or with paymentservices used by plurality of merchants; retrieving website data from awebsite of a first merchant from the plurality of merchants;determining, based on the retrieved website data, whether any termsappearing on the website of the first merchant match the terms in thedictionary of terms; selecting, based on the determining whether anyterms appearing on the website of the first merchant match the terms inthe dictionary of terms, one or more payment provider services toadvertise to the first merchant; training an advertisement responsemodel using a training data set that indicates a respective response toan advertising campaign for each unit in the training data set, whereinunits of the training data set correspond to a first subset of theplurality of merchants; determining, based on the training theadvertisement response model using the training data set, that a firstaccuracy value corresponding to the advertisement response model is lessthan an accuracy value threshold; determining, by the computer using thedictionary of terms, a target data set that is different from thetraining data set, wherein units of the target data set correspond to asecond subset of the plurality of merchants, the second subset beingdifferent from the first subset; identifying, based on executing theadvertisement response model using the target data set, one or moreunits from the target data set for which to run the advertisingcampaign; receiving one or more responses corresponding to a run of theadvertising campaign with respect to the identified one or more unitsfrom the target data set; updating the training data set based on theone or more responses; and determining, based on training theadvertisement response model using the updated training data set,whether a second accuracy value corresponding to the advertisementresponse model is greater than the accuracy value threshold.
 9. Thenon-transitory computer readable medium of claim 8, wherein theoperations further comprise: determining that the second accuracy valueis greater than or equal to the accuracy value threshold; adjusting thetarget data set by removing the units included in the identified one ormore units from the target data set; and based on executing theadvertisement response model with respect to the adjusted target dataset, determining one or more units of the adjusted target data set thathave a positive predicted response to the advertising campaign.
 10. Thenon-transitory computer readable medium of claim 9, wherein theoperations further comprise: transmitting an advertisement to one ormore devices associated with the determined one or more units of theadjusted target data set.
 11. The non-transitory computer readablemedium of claim 8, wherein each unit of the identified one or more unitsof the target data set is associated with a certainty score that is lessthan a certainty score threshold.
 12. The non-transitory computerreadable medium of claim 8, wherein the second accuracy value is greaterthan the first accuracy value.
 13. A system, comprising: anon-transitory memory; and one or more hardware processors coupled tothe non-transitory memory and configured to read instructions from thenon-transitory memory to cause the system to perform operationscomprising: electronically scanning websites of a plurality ofmerchants; determining a frequency of usage of one or more terms on thewebsites, a placement of the one or more terms on the websites, or avisual appearance of the one or more terms on the websites; determining,based on the frequency, the placement, or the visual appearance of theone or more terms on the websites, a dictionary of terms associated withthe websites; correlating the terms in the dictionary of terms withtypes of merchants from the plurality of merchants or with paymentservices used by plurality of merchants; retrieving website data from awebsite of a first merchant from the plurality of merchants;determining, based on the retrieved website data, whether any termsappearing on the website of the first merchant match the terms in thedictionary of terms; selecting, based on the determining whether anyterms appearing on the website of the first merchant match the terms inthe dictionary of terms, one or more payment provider services toadvertise to the first merchant; training an advertisement responsemodel using a training data set that indicates a respective response toan advertising campaign for each unit in the training data set, whereinunits of the training data set correspond to a first subset of theplurality of merchants; determining, based on the training theadvertisement response model using the training data set, that a firstaccuracy value corresponding to the advertisement response model is lessthan an accuracy value threshold; determining, using the dictionary ofterms, a target data set that is different from the training data set,wherein units of the target data set correspond to a second subset ofthe plurality of merchants, the second subset being different from thefirst subset; identifying, based on executing the advertisement responsemodel using the target data set, one or more units from the target dataset for which to run the advertising campaign; receiving one or moreresponses corresponding to a run of the advertising campaign withrespect to the identified one or more units from the target data set;updating the training data set based on the one or more responses; anddetermining, based on training the advertisement response model usingthe updated training data set, whether a second accuracy valuecorresponding to the advertisement response model is greater than theaccuracy value threshold.
 14. The system of claim 13, wherein theoperations further comprise: determining that the second accuracy valueis greater than or equal to the accuracy value threshold; adjusting thetarget data set by removing the units included in the identified one ormore units from the target data set; and based on executing theadvertisement response model with respect to the adjusted target dataset, determining one or more units of the adjusted target data set thathave a positive predicted response to the advertising campaign.
 15. Thesystem of claim 14, wherein the operations further comprise:transmitting an advertisement to one or more devices associated with thedetermined one or more units of the adjusted target data set.
 16. Thesystem of claim 13, wherein the operations further comprise: rankingeach unit of the target data set according to a respective certaintyscore for each unit of the target data set, wherein each of theidentified one or more units from the target data set is associated witha lower respective certainty score than respective certainty scores ofthe other units in the target data set.
 17. The system of claim 13,wherein each unit of the identified one or more units of the target dataset is associated with a certainty score that is less than a certaintyscore threshold.
 18. The system of claim 13, wherein the second accuracyvalue is greater than the first accuracy value.
 19. The system of claim13, wherein the operations further comprise removing the identified oneor more units from the target data set.
 20. The system of claim 13,wherein the advertisement response model comprises one or more of:AdaBoost classifier, logistic regression, support vector machines,gradient boosting classifier, Isolation Forest, linear discriminantanalysis, neural networks, decision trees classifier, Random Forestclassifier, nave Bayesian classifier, or Bagging classifier.